Background modeling by subspace learning on spatio-temporal patches

This paper presents a novel background model for video surveillance—Spatio-Temporal Patch based Background Modeling (STPBM). We use spatio-temporal patches, called bricks, to characterize both the appearance and motion information. Our method is based on the observation that all the background bricks at a given location under all possible lighting conditions lie in a low dimensional background subspace, while bricks with moving foreground are widely distributed outside. An efficient online subspace learning method is presented to capture the subspace, which is able to model the illumination changes more robustly than traditional pixel-wise or block-wise methods. Experimental results demonstrate that the proposed method is insensitive to drastic illumination changes yet capable of detecting dim foreground objects under low contrast. Moreover, it outperforms the state-of-the-art in various challenging scenes with illumination changes. 2012 Elsevier B.V. All rights reserved.